Predicting Rankings of Software Verification Competitions
نویسندگان
چکیده
Soware verication competitions, such as the annual SV-COMP, evaluate soware verication tools with respect to their eectivity and eciency. Typically, the outcome of a competition is a (possibly category-specic) ranking of the tools. For many applications, such as building portfolio solvers, it would be desirable to have an idea of the (relative) performance of verication tools on a given verication task beforehand, i.e., prior to actually running all tools on the task. In this paper, we present a machine learning approach to predicting rankings of tools on verication tasks. e method builds upon so-called label ranking algorithms, which we complement with appropriate kernels providing a similarity measure for verication tasks. Our kernels employ a graph representation for soware source code that mixes elements of control ow and program dependence graphs with abstract syntax trees. Using data sets from SV-COMP, we demonstrate our rank prediction technique to generalize well and achieve a rather high predictive accuracy. In particular, our method outperforms a recently proposed feature-based approach of Demyanova et al. (when applied to rank predictions).
منابع مشابه
Evaluating Software Verification Systems: Benchmarks and Competitions
This report documents the program and the outcomes of Dagstuhl Seminar 14171 “Evaluating Software Verification Systems: Benchmarks and Competitions”. The seminar brought together a large group of current and future competition organizers and participants, benchmark maintainers, as well as practitioners and researchers interested in the topic. The seminar was conducted as a highly interactive ev...
متن کاملEvaluating Software Verification Systems: Benchmarks and Competitions (Dagstuhl Reports 14171)
متن کامل
LMPP: A Large Margin Point Process Combining Reinforcement and Competition for Modeling Hashtag Popularity
Predicting the popularity dynamics of Twitter hashtags has a broad spectrum of applications. Existing works have primarily focused on modeling the popularity of individual tweets rather than the underlying hashtags. As a result, they fail to consider several realistic factors contributing to hashtag popularity. In this paper, we propose Large Margin Point Process (LMPP), a probabilistic framewo...
متن کاملCompetition on Software Verification - (SV-COMP)
This report describes the definitions, rules, setup, procedure, and results of the 1st International Competition on Software Verification. The verification community has performed competitions in various areas in the past, and SV-COMP’12 is the first competition of verification tools that take software programs as input and run a fully automatic verification of a given safety property. This yea...
متن کاملApplication of PageRank Model for Olympic Women’s Taekwondo Rankings: Comparison of PageRank and Accumulated Point Index System
Background. Although the World Taekwondo federation currently applies the APIS ranking method to calculate the Olympic rankings, some limitations exist. Objectives. This study applies the PageRank model to Olympics Taekwondo rankings. Methods. The 2015-2018 World Taekwondo Grand Prix competition results for women’s four weight classes (-49kg, -57kg, -67kg, +67kg) were used as research data, t...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1703.00757 شماره
صفحات -
تاریخ انتشار 2017